---
title: "Balanced messaging about solar geoengineering does not reduce average support for emissions reductions"  
execute:
  echo: false
  message: false
  warning: false
format:
  jasa-pdf:
    keep-tex: true  
    journal:
      blinded: false
    include-in-header:
      - file: packages.tex  
date: last-modified
author:
  - name: |
          | Damian Antoan, Nicholas Chiang, Spencer Dearman, Santiago Espejel, 
          | Manareldeen Fajors, Darina Huang, Elliott Husseman, Justin Lavigne, 
          | Isabella Lin, Neel Maheshwari, Zidane Marinez, George Nottley, 
          | Julien Perce,  Ezra Santos, Natasha Sokhi, Carlos Sole, Adam Tuchler, 
          | Jay Udinsky, Zikai Li, and Andrew C. Eggers
    affiliations: University of Chicago 
    acknowledgements: |
      Eggers is Professor of Political Science at the University of Chicago. Li is a PhD Candidate in Political Science at the University of Chicago. Other authors were undergraduate students in "Social Science Inquiry" at the University of Chicago in Spring of 2024. Research was supported by a grant from the University of Chicago's Climate Systems Engineering Initiative (CSEi) and by a University of Chicago Core Conversation Innovation Grant. We thank Molly Offer-Westort for feedback. Anonymized pre-analysis plan ("Effects of informing Americans about solar geoengineering") available at <https://osf.io/yf4vs/?view_only=8d21d57844cc421280b1e99d12f7caad>. See Appendix @tbl-pre-reg-report for information about the correspondence between the manuscript and the pre-registration document. The University of Chicago SBS-IRB issued an exempt determination for this research (IRB23-0386-AM003). 
abstract: |
  Solar geoengineering offers a speculative means to cool the planet by reflecting solar radiation into space. While some research suggests that awareness of solar geoengineering could reduce public support for decarbonization through a moral hazard mechanism, other studies indicate that it could serve as a "clarion call" that motivates further action. Using a pre-registered factorial design, we assess how sharing balanced information on solar geoengineering affects attitudes toward decarbonization policies and climate attitudes among 2,509 US residents. We do not find that solar geoengineering information affects support for decarbonization on average, though it may increase support among initially less supportive subgroups; moreover, this information tends to increase the perception that climate change is a daunting problem that cannot be resolved without decarbonization. Our results suggest that concerns about moral hazard should not discourage research on solar geoengineering---as long as the public encounters realistic messages about solar geoengineering’s role.
  
  Word count: 3956
bibliography: sai.bib  
linestretch: 1.5
fig-cap-location: top

---


\spacingset{1}  <!-- undoing the double spacing -->
\doublespacing  <!-- redoing in a way that does not double space the tables --> 
\setlength{\parindent}{12pt}  <!-- paragrpah indents --> 
\setlength{\parskip}{0pt} <!-- 6pt plus 2pt minus 1pt}} --> 

## Introduction

Solar geoengineering refers to a group of proposed techniques that aim to reduce global warming by reflecting a portion of sunlight away from Earth. One prominent proposal is stratospheric aerosol injection (SAI), which mimics the cooling effect of volcanic eruptions by injecting reflective aerosols such as sulfur dioxide into the upper atmosphere to scatter sunlight [@national2021reflecting]. Climate scientists remain unsure how a sustained program of SAI would affect the ozone layer and local weather patterns; questions also remain about the potential health and environmental effects of spraying sulfuric acid or other aerosols into the atmosphere. 

As solar geoengineering research has attracted more attention, critics have raised a further, more social scientific objection: will researching or even discussing solar geoengineering undermine public support for the vital work of decarbonization?[^donotdiscuss] If the public learns that solar geoengineering can lessen the damage from climate change, there might be a weakening of public support for taking the necessary steps to reduce the concentration of greenhouse gases in the atmosphere, exacerbating the underlying problem and some of its consequences (such as ocean acidification). @keith2000geoengineering refers to this possibility as “moral hazard;” others refer to crowding out [@cherry2021does], lack of self-control [@wagner2015climate], or risk compensation [@lin2013does;@reynolds2015critical]. Even if solar geoengineering is found to be effective and safe in engineering terms, deploying it <!-- changed solar geoengineering to "it" here -->could have a net negative impact if this moral hazard mechanism operates strongly enough. In that case, decarbonization should be the sole focus and investigation of solar geoengineering  (let alone deployment) should be discouraged. 

[^donotdiscuss]: For example, see @thompson2021sierra and @hamilton2013earthmasters. The same question arises in discussions of adaptation [@lin2013does] and carbon dioxide removal [@hart2022moral]. <!--  Joanna Thompson, "Solar Geoengineering Could Become the `New Form of Denial' for Silicon Valley Enthusiasts, Critics Say", *Sierra: the magazine of the Sierra Club*, September 21, 2023. [(link)](https://www.sierraclub.org/sierra/solar-geoengineering-could-become-new-form-denial-silicon-valley-enthusiasts-critics-say); Clive Hamilton, *Earthmasters*, Allen & Unwin, 2013. --> 

Although some previous research has validated concerns about SAI and moral hazard [@raimi2019framing;@campbell-ervai2017influence;@andrews2022anticipating], other studies suggest that informing the public about SAI can instead serve as a "clarion call" that inspires greater support for decarbonization [@merk2016knowledge; @cherry2021does], as we discuss below. 
Given these conflicting results and important limitations in previous work, we revisit the issue with a new study that builds on previous work while addressing some of its shortcomings. Our pre-registered survey experiment assesses how exposing Americans to balanced information about solar geoengineering, either on its own or in conjunction with a message about decarbonization, affects support for mitigation policy and attitudes toward climate change. We also assess how these effects vary with respondents' political ideology, concern about global warming, and attribution of global warming to human action (all measured pre-treatment).

We find no evidence that informing Americans about SAI affects average support for emissions reductions, even compared to a message emphasizing the importance of decarbonization.  
We also do not find that SAI information has more positive effects for liberal respondents, but we do find suggestive evidence that it increases support for mitigation policy among respondents who are less concerned about climate change. Turning to attitudes about climate change, we find that providing balanced SAI information increases average concern about global warming (though not when paired with the decarbonization message) and increases the perception that without emissions reductions we may end up “deploying unproven technologies with harmful consequences” to address global warming; it also decreases respondents’ confidence that technology will allow us to address global warming without emissions reductions. Together, our results suggest that informing people about SAI may make climate change seem like a more challenging, higher-stakes problem, especially among those who are initially relatively complacent about climate change. 


## Background 

Theoretical considerations and prior empirical evidence yield conflicting expectations about how sharing information about SAI and related technologies might shape public attitudes toward climate change mitigation. 

The moral hazard hypothesis suggests that informing the public about SAI could lead to an unwarranted decline in public support for decarbonization. Offered a temporary means of limiting global warming such as SAI, the public may come to view costly emissions reductions as unnecessary.  Consistent with this moral hazard hypothesis, @raimi2019framing found that reading a highly optimistic description of SAI could make Americans less supportive of policies that would reduce carbon emissions. Similarly, @campbell-ervai2017influence report that learning about carbon dioxide removal may reduce support for emissions reductions by diminishing the perceived threat of climate change.[^mediation] @andrews2022anticipating find via incentivized games that subjects who are asked to make policy decisions expect others to engage in moral hazard behavior, discouraging the deployment of geoengineering technologies. Considering that SAI experts view SAI as a complement to aggressive emissions reductions [@keith2021leastbad;@national2021reflecting], it is important to assess this moral hazard mechanism as one of the possible harmful unintended consequences of further research into SAI. 

[^mediation]: This result comes from mediation analysis that relies on strong assumptions; the reported effect of information on policy support is not statistically significant.  

SAI information could have the opposite effect, however, increasing desire to reduce emissions. As @merk2016knowledge noted, learning about SAI could act as a "clarion call." The knowledge that experts are seriously considering spraying sulfuric acid into the atmosphere to block the sun’s rays could reinforce the belief that climate change is a serious problem. To the extent that SAI sounds like a risky technology that should be avoided, information about SAI may also make recipients more committed to reducing emissions as a way of making sure that SAI is never attempted. Consistent with this, @merk2016knowledge found that providing information on SAI increased German survey respondents’ average willingness to pay for carbon offsets. Similarly, in a US sample @cherry2021does found that providing information about solar geoengineering increased support for a carbon tax among respondents with egalitarian and communitarian worldviews. 

Of course, SAI information could also have little average effect on attitudes toward decarbonization, whether because these attitudes are relatively fixed or because such information provokes a mix of conflicting reactions. Indeed, most recent studies appear to show minimal impacts of geoengineering information. In a US sample @schoenegger2024moral found that providing solar geoengineering information had no significant effect on support for emissions reductions measured by either behavioral choices or stated preferences. Similarly, @fairbrother2016geoengineering found that solar geoengineering information does not significantly impact trust in climate science or willingness to pay taxes on polluting energy in a UK sample, and @merk2024presenting found no effect of including solar geoengineering information in appeals for climate action on Facebook, except when in conjunction with extreme messaging. @austin2021in found no significant effect of reading a news article about solar geoengineering on commitment to mitigation efforts in a US sample, and @andrews2022anticipating also reports no evidence of moral hazard among participants.[^co2fn]  

[^co2fn]: @hart2022moral and @sloot2024role also show minimal effects of information about carbon dioxide removal on support for emissions reductions.




## Our contribution

Our research seeks to contribute to the existing literature on effects of SAI information in three principal ways.

First, while other studies [@merk2016knowledge; @fairbrother2016geoengineering; @cherry2021does; @schoenegger2024moral] examine the effect of providing SAI information in isolation, we use a factorial design that allows us to compare the effect of providing SAI information to the effect of providing a decarbonization message, and to compare the effect of providing SAI information on its own to the effect of providing both messages. Critics of SAI [e.g. @thompson2021sierra; @vasquez2023sierra] stress that we should keep a laser focus on decarbonization, while SAI researchers describe SAI as a possible complement to deep decarbonization [e.g. @keith2021leastbad; @national2021reflecting]. Our design allows us to compare the effects of an SAI information on its own, a decarbonization message on its own, and a combined message.  


Second, while previous studies [particularly @raimi2019framing; @merk2024presenting] measured the effects of framing SAI in exaggeratedly positive or negative terms,[^swissmix] we focus on reactions to a neutral, balanced presentation of both SAI and decarbonization. @raimi2019framing established that describing SAI in highly optimistic terms (suggesting "we wouldn’t have to do much more" to mitigate climate change) could reduce support for decarbonization among Americans; a more moderate framing in the same study did not produce such an effect. Building on this more moderate framing (and similar to the framing in @schoenegger2024moral), we present SAI much as it is presented by SAI experts [@keith2021leastbad] -- as a potentially effective medium-term measure with acknowledged risks. Distinct from @raimi2019framing's framings, our framing refrains from explicitly depicting SAI as either a substitute or a complement to emissions reductions, thus leaving it to respondents to draw their own conclusions about the implications of SAI for decarbonization efforts. 


[^swissmix]: In @wenger2021public, Swiss respondents are skeptical of the idea that geoengineering could make emissions reductions unnecessary.


Third, we take important steps to advance the scientific rigor of studies on this topic. To our knowledge, ours is the first study on the effects of SAI information to be pre-registered, and after @raimi2019framing it will be the second to make its data publicly available on publication. Moreover, in two important studies in this literature [@merk2016knowledge; @cherry2021does], the main regression analyses adjust for post-treatment variables, which could undermine the randomization and produce biased results [@montgomery2018conditioning]. By contrast, all control variables and moderators in our analysis are measured before treatment was applied. 


```{r}
#| echo: false
#| message: false
library(tidyverse)
load("data/D_and_D_post_exclusions_v2.RData") # see ssi_2024_construct_analysis_data.qmd
```


## Survey design

In this section we explain the design of our survey. The exact questions appear in the Appendix. 

### Treatments 

All respondents were shown a screen explaining how carbon emissions contribute to global warming (see @fig-control-message in Appendix). Our message combines a "blanket metaphor" explanation of the greenhouse effect (from @bergquist2022information) with a statement that past carbon emissions will continue to warm the planet for hundreds or even thousands of years (derived from @keith2021leastbad, who calls this "the single most important fact about climate change").     

Respondents were then randomly assigned into one of four groups: (1) Control; (2) Decarbonization; (3) SAI; (4) Both. Respondents in the control group were not given any additional information. The Decarbonization group was shown the message at left in @fig-treatments. It states that reducing emissions is essential, lists "key steps to take", and shows an image of solar panels and wind turbines. The SAI group was shown the message at right in @fig-treatments. It states that emissions reductions will not immediately repair the atmosphere, so "scientists are researching other ways to cool the planet." It then describes SAI and some of its risks and benefits, accompanied by a diagram showing sunlight being reflected into space by stratospheric aerosols. Respondents in the Both group saw the Decarbonization information followed by the SAI information.[^whyordered]


<!-- FIGURE 1 --> 

::: {#fig-treatments layout-ncol=2}

![](images/decarbonization.png)

![](images/sai.png)

Treatments shown to survey participants: Decarbonization (left) and SAI (right)
:::


[^whyordered]: Presented in this order, the Both treatment roughly mirrors @keith2021leastbad's argument in favor of SAI research. Respondents in the Both condition see more text than those in SAI or Decarbonization, and those in Control see less. 


### Outcome measures 

To measure support for a range of emissions reduction policies, we ask our respondents to assess eight possible US climate policies drawn from @raimi2019framing (international emissions treaty, stricter fuel efficiency standards, subsidies to renewable energy, a carbon tax, expanded nuclear power, energy conservation regulations, industrial emissions reductions, clean energy research subsidies) at the very end of the survey.  We convert support for each measure to a numerical 1-5 scale and average them to obtain an index of post-treatment support for emissions reduction policy.

To assess mechanisms by which SAI information could affect policy preferences, we asked additional questions after treatment but before the policy questions.[^mechtiming] To tap into the “clarion call” mechanism highlighted by @merk2016knowledge, we asked respondents to state how much (five points between “Not at all” and “A great deal”) they think global warming will affect them personally and how much it will negatively affect future generations. We convert the two responses to a numerical 1-5 scale and average them to obtain an index of post-treatment global warming concern.

[^mechtiming]: These questions raise the salience of particular concerns (e.g.\ the fate of future generations, risks from unproven technologies), which may moderate the effect of SAI information on policy support given that our policy questions appear at the end of the survey.  

To assess other mechanisms through which SAI information might affect policy preferences, we surveyed respondents' agreement with each of the following statements:

1. Techno-threat: “If we don't take action to reduce emissions now, then responding to global warming in the future is likely to require deploying unproven technologies with harmful consequences.” (SAI information could make salient a threat that respondents would seek to avoid, as suggested by @merk2016knowledge.)
2. Belief in the efficacy of emissions reductions: “Humanity could limit future increases in the global average temperature by reducing emissions of carbon dioxide now.” (SAI information could strengthen respondents’ understanding of and belief in the greenhouse effect.) 
3. Techno-fix: "Thanks to technology, there is a good chance that we will be able to completely solve the problem of global warming in the next 50 years without significantly reducing carbon dioxide emissions." (SAI information could convince respondents that technology will solve climate change without the need for emissions reductions.)


### Pre-treatment covariates

At the start of the survey, we asked respondents to assess their political ideology on a five-point scale from "Very liberal" to "Very conservative." After reading our explanation of global warming, but before seeing any treatment messages, all respondents were also asked how concerned they were about global warming, how much of global warming they think is caused by human activities, and whether they believe that they can "do something to address global warming." All four pre-treatment questions were used as categorical prognostic covariates in regression analysis, as explained below.


### Sample

We fielded our survey to 2,509[^power] US residents[^whyUS] on April 30, 2024, using CloudResearch Connect, an online survey panel. To counteract typical biases in internet panels, we specified quotas to obtain a 50-50 gender split and a 30-30-40 split between respondents who identify as Democrats, Republicans, and other (Independents, something else, prefer not to say).  

[^power]: The sample size was determined by our survey budget and guided by a priori power calculations, which indicated we would have sufficient power (above 0.8) to detect an effect of 0.15 of a standard deviation in the outcome variable (see Appendix @sec-sample-size for more details).

[^whyUS]: We study the US because it will likely have an important role in solar geoengineering, but similar research in other populations is important given solar geoengineering's global scope [e.g. @sugiyama2020north].


```{r exclusion-numbers}
D |> 
  group_by(Condition) |> 
  mutate(outlier = log(`Duration (in seconds)`) > quantile(log(`Duration (in seconds)`), .75) + 1.5*IQR(log(`Duration (in seconds)`)) | log(`Duration (in seconds)`) < quantile(log(`Duration (in seconds)`), .25) - 1.5*IQR(log(`Duration (in seconds)`))) -> D_with_outliers 

number_of_outliers <- D_with_outliers |> ungroup() |> summarize(sum(outlier)) |> as.numeric()

D_with_outliers |>
  filter(!outlier) |> 
  mutate(attention_check_status = case_when(attention_check == "Strongly disagree,Neither agree nor disagree" ~ "Passed",
                                            !str_detect(attention_check, "\\,") ~ "Failed: single response",
                                            TRUE ~ "Failed: more than one response")) -> D_woo_ac_coded

number_of_inattentives <- D_woo_ac_coded |> ungroup() |> summarize(sum(attention_check_status != "Passed"))
```

Our pre-analysis plan stated that we would exclude respondents who completed the survey unusually quickly or slowly, conditional on treatment status (a total of `r number_of_outliers` respondents), as well as those who missed an attention check question (an additional `r number_of_inattentives` respondents), given satisfactory balance in the post-exclusion sample. To allay concerns about post-treatment bias caused by these exclusions (and to meet the journal's submission guidelines), we deviate from this plan and report results for the full sample in the main text; all pre-registered results using the restricted sample appear in the Appendix (along with balance tests). The two sets of results are substantively identical.[^exclusionresults] 

[^exclusionresults]: Some marginally significant results become marginally insignificant and vice versa, but the point estimates are very similar.



Our sample is broadly representative of the US in terms of gender, partisanship, and race. It is younger than the population (with e.g.\ about 32% of respondents in their thirties, compared to around 18% of the population, and 3% of respondents over 70, compared to 15% of the population); it is also more educated, with about 40% holding BAs compared to about 22% of the population, and its distribution of household income is lower than that in the population. (See Appendix @fig-rep for further details on representativeness.)    


## Results 

In all analysis, we use OLS regression with heteroskedasticity-robust (HC2) standard errors. Analysis of pilot data showed that all four pre-treatment variables (ideology, concern about global warming, attribution of global warming, and self-efficacy) predict support for climate mitigation policy, so following our pre-analysis plan we include all four variables as prognostic covariates. Also following our pre-analysis plan, we standardize all outcome variables  so that all treatment effects are measured in standard deviations of the outcome variable. 

### Manipulation check

We expected that few respondents would be familiar with SAI and that the SAI information we provide would therefore substantially increase awareness of SAI. To check this, we asked respondents to indicate whether each of four technologies is viewed by some experts as a promising response to global warming. Two of the listed technologies are fanciful ("Pumping dry ice into the earth's crust" and "Extracting silicates from agricultural soil") while one describes SAI (“Releasing sulfur dioxide into the stratosphere”) and one is a component of decarbonization ("Converting heating systems to use electricity instead of gas"). Consistent with expectations,  we find  that about 70% of respondents who receive the SAI information indicate that “Releasing sulfur dioxide into the stratosphere” is considered promising by some experts, compared to only about 20% of respondents who did not receive this information. By contrast, we do not find an effect of providing the Decarbonization treatment on respondents' recognition that electrifying heating systems is seen as promising. (Full results appear in Appendix @fig-manip-check and @tbl-manip-check.) This suggests that (i) our respondents were mostly unaware of SAI before the experiment, (ii) many of those who received SAI information became aware of the technology as a result, and (iii) awareness of decarbonization/electrification is higher at baseline and was unaffected by our information treatments.   

### Effects of information on support for mitigation policy


```{r spi-regs}
#| echo: false

spi_reg_list_given_data <- function(this_data = D_post_exclusions){
  spi_SAI <- estimatr::lm_robust(standardized_policy_index ~ SAI_either + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)

  spi_Emissions <- estimatr::lm_robust(standardized_policy_index ~ Emissions_either + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)

  spi_all_four <- estimatr::lm_robust(standardized_policy_index ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  
  list(spi_SAI, spi_Emissions, spi_all_four)

}

spi_reg_list_wo_exclusions <- spi_reg_list_given_data(D)
```

<!-- Combine results and plot  -->

```{r}

spi_reg_results_wo_exclusions <- rbind(
  spi_reg_list_wo_exclusions[[1]] |> estimatr::tidy() |> filter(term =="SAI_either") |> 
    mutate(Model ="1"), 
  spi_reg_list_wo_exclusions[[2]] |> estimatr::tidy() |> filter(term =="Emissions_either") |> 
    mutate(Model = "2"), 
    spi_reg_list_wo_exclusions[[3]] |> estimatr::tidy() |> filter(str_detect(term, "Condition")) |> 
    mutate(Model ="3"))

df <- spi_reg_results_wo_exclusions


df <- df |>
  mutate(term = recode(term,
    "SAI_either"           = "Provided SAI info",
    "Emissions_either"     = "Provided decarbonization info",
    "ConditionEmissions"   = "Provided decarbonization info only",
    "ConditionSAI"         = "Provided SAI info only",
    "ConditionBoth"        = "Provided both SAI and decarbonization info"
  ))


df <- df |>
  mutate(order = case_when(
    term == "Provided SAI info"                   ~ 3.7,
    term == "Provided decarbonization info"         ~ 2.7,
    term == "Provided decarbonization info only"    ~ 1.6,
    term == "Provided SAI info only"                ~ 1.3,
    term == "Provided both SAI and decarbonization info" ~ 1
  ))


plt <- ggplot(df, aes(x = estimate, y = order, color = Model, shape = Model)) +
  geom_vline(xintercept = 0, color = "gray50", linetype = 2) +
  geom_errorbarh(aes(xmin = conf.low, xmax = conf.high), height = 0.2) +
  geom_point(size = 3, fill = "white") +
  scale_color_manual(values = c("1" = "#3B9AB2", "2" = "#E1AF00", "3" = "#F98400")) +
  # Use the order variable for the y-axis and assign the recoded term labels
  scale_y_continuous(breaks = df$order,
                     labels = df$term,
                     expand = expansion(mult = c(0.1, 0.1))) +
  labs(x = "Estimates with 95% Confidence Intervals",
       y = "") +
  # theme(text = element_text(size = 10.5, family = "CMU Concrete"),
  #       axis.title = element_text(family = "CMU Concrete"),
  #       plot.title = element_text(family = "CMU Concrete"),
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black"))
```

```{r}
#| echo: false
#| fig-width: 6
#| fig-height: 4
#| label: fig-main-results
#| fig-cap: "Effect of information treatments on support for emission reduction policies"

## FIGURE 2
plt
```

<!-- we should probably move table results to appendix? -->

@fig-main-results reports estimated effects of our information treatments on our index of support for emissions mitigation policy.[^table-too] In model 1 we compare respondents who received SAI information (whether or not they also received the decarbonization information) to those who did not; in model 2 we compare respondents who received  decarbonization information (whether or not they also received the SAI information) to those who did not; in model 3 we regress the outcome on the categorical treatment condition, thus comparing respondents across the four treatment conditions in our factorial experiment.


[^table-too]: Appendix @tbl-main-results shows the corresponding regression table. 

```{r}
# compute $p$-values for other comparisons
get_the_variance_for_a_diff <- function(mod, var1, var2){
  vc <- vcov(mod)[c(var1, var2), c(var1, var2)]
  vc[1,1] + vc[2,2] - 2*vc[1,2]
}
# get_the_variance(swi_condition, "ConditionBoth", "ConditionSAI")
get_the_p_for_a_diff <- function(mod, var1, var2){
  v <- get_the_variance_for_a_diff(mod, var1, var2)
  diff <- coef(mod)[var1] - coef(mod)[var2]
  as.numeric(2*(1 - pnorm(abs(diff)/sqrt(v))))
}
# get_the_p_for_a_diff(swi_condition, "ConditionBoth", "ConditionSAI")
# get_the_p_for_a_diff(swi_condition, "ConditionBoth", "ConditionSAI")
```

All the point estimates are positive, but none of the effects is significant at the .05 level, and magnitudes are small: e.g., the point estimate in model 3 suggests that providing both SAI and decarbonization information increases support for mitigation policy by `r round(coefficients(spi_reg_list_wo_exclusions[[3]])["ConditionBoth"] ,3)` standard deviations compared to no information ($p$ =`r round(summary(spi_reg_list_wo_exclusions[[3]])$coefficients["ConditionBoth", "Pr(>|t|)"], 3)`).

In the Appendix, we present exploratory analysis assessing the effect of our information treatments on each of the policies in the index (@fig-sai-on-individual-outcomes, @fig-info-on-individual-outcomes). We find significant positive effects of the decarbonization message, and marginally significant effects of the SAI message, on support for building nuclear power plants, as well as marginally significant positive effects of all messages on support for encouraging individuals to use less energy. Future confirmatory analysis could assess these effects. 

Appendix @tbl-main-results-weighted shows a version of the analysis in @fig-main-results  where we weight the sample to match population demographic margins [@lumley2017fitting].[^weighted_reg_no_pap] As in the unweighted results, effects are null (though the point estimate of the effect of the SAI message is larger in the weighted sample).

[^weighted_reg_no_pap]: This analysis was not in our PAP but was suggested by a reviewer. 



### Heterogeneous effects of SAI information on support for mitigation policy

Although SAI information did not significantly affect average support for mitigation policy (@fig-main-results), it may have larger effects in subgroups. Following our PAP, we assessed treatment effect heterogeneity by respondents' political ideology (prompted by @cherry2021does's finding that SAI information increased carbon tax support among more egalitarian and communitarian respondents) and by respondents' concern about climate change and attribution of climate change to human causes (because we hypothesized that the "clarion call" effect would be larger for people who are initially more complacent about climate change). @fig-het-main shows the results.

```{r main-effect-heterogeneity}
#| echo: false

spi_het_reg_list_given_data <- function(this_data){
  spi_SAI_by_affiliation <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*I(Affiliation %in% c("Very liberal", "Liberal")) + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  spi_SAI_by_concern <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*I(cc_concern_pre %in% c("Not concerned at all", "Slightly concerned")) + Affiliation + attribution_quintile + self_efficacy_pre, data = this_data)
  spi_SAI_by_attribution <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*I(attribution_quintile %in% c(1, 2)) + Affiliation + cc_concern_pre + self_efficacy_pre, data = this_data)

  list("By ideology" = spi_SAI_by_affiliation, "By concern" = spi_SAI_by_concern, "By attribution" = spi_SAI_by_attribution)
}
het_reg_table_given_reg_list <- function(reg_list){
  rows <- tribble(~term, ~term, ~term, ~term,
                 "Covariates", "Yes", "Yes", "Yes")
  attr(rows, "position") <- 17 # put it at the bottom

  modelsummary::modelsummary(reg_list, 
                           stars = T, 
                           gof_map = c("nobs", "r.squared", "adj.r.squared"), 
                           coef_map = c("(Intercept)" = "Intercept", 
                                        "SAI_either" = "Provided SAI info", 
                                        'I(Affiliation %in% c("Very liberal", "Liberal"))TRUE' = "Liberal",
                                        'SAI_either:I(Affiliation %in% c("Very liberal", "Liberal"))TRUE' = "SAI info X Liberal",
                                        'I(cc_concern_pre %in% c("Not concerned at all", "Slightly concerned"))TRUE' = "Less concerned about global warming", 
                                        'SAI_either:I(cc_concern_pre %in% c("Not concerned at all", "Slightly concerned"))TRUE' = "SAI info X Less concerned",
                                        "I(attribution_quintile %in% c(1, 2))TRUE" = "Attributes less to human action", 
                                        "SAI_either:I(attribution_quintile %in% c(1, 2))TRUE" = "SAI info X Attributes less to human action"), 
                           add_rows = rows) |>
  tinytable::group_tt(j = list("DV: Index of support for mitigation policy" = 2:4))
}

spi_het_reg_list_wo_exclusions <- spi_het_reg_list_given_data(D)

```


```{r}

get_the_variance_for_a_sum <- function(mod, var1, var2){
  vc <- vcov(mod)[c(var1, var2), c(var1, var2)]
  vc[1,1] + vc[2,2] + 2*vc[1,2]
}
spi_het_reg_results_wo_exclusions <- rbind(
  spi_het_reg_list_wo_exclusions[[1]] |> estimatr::tidy() |> mutate(Model = "1"), 
  spi_het_reg_list_wo_exclusions[[2]] |> estimatr::tidy() |> mutate(Model = "2"), 
  spi_het_reg_list_wo_exclusions[[3]] |> estimatr::tidy() |> mutate(Model = "3")
) |>
  # Filter to only the terms we want to recode
  filter(term %in% c(
    "SAI_either", 
    'SAI_either:I(Affiliation %in% c("Very liberal", "Liberal"))TRUE', 
    'SAI_either:I(cc_concern_pre %in% c("Not concerned at all", "Slightly concerned"))TRUE', 
    "SAI_either:I(attribution_quintile %in% c(1, 2))TRUE"
  )) 

  
spi_het_reg_results_wo_exclusions[spi_het_reg_results_wo_exclusions$term =='SAI_either:I(Affiliation %in% c("Very liberal", "Liberal"))TRUE' & spi_het_reg_results_wo_exclusions$Model == "1","estimate"]  = spi_het_reg_results_wo_exclusions[
  spi_het_reg_results_wo_exclusions$term =='SAI_either:I(Affiliation %in% c("Very liberal", "Liberal"))TRUE'& spi_het_reg_results_wo_exclusions$Model == "1","estimate"] + 
  spi_het_reg_results_wo_exclusions[spi_het_reg_results_wo_exclusions$term =="SAI_either"& spi_het_reg_results_wo_exclusions$Model == "1","estimate"]

spi_het_reg_results_wo_exclusions[spi_het_reg_results_wo_exclusions$term =='SAI_either:I(cc_concern_pre %in% c("Not concerned at all", "Slightly concerned"))TRUE'& spi_het_reg_results_wo_exclusions$Model == '2','estimate']  = spi_het_reg_results_wo_exclusions[
  spi_het_reg_results_wo_exclusions$term =='SAI_either:I(cc_concern_pre %in% c("Not concerned at all", "Slightly concerned"))TRUE'& spi_het_reg_results_wo_exclusions$Model == '2','estimate'] + 
  spi_het_reg_results_wo_exclusions[spi_het_reg_results_wo_exclusions$term =='SAI_either'& spi_het_reg_results_wo_exclusions$Model == '2','estimate']

spi_het_reg_results_wo_exclusions[spi_het_reg_results_wo_exclusions$term =='SAI_either:I(attribution_quintile %in% c(1, 2))TRUE'& spi_het_reg_results_wo_exclusions$Model == '3','estimate']  = spi_het_reg_results_wo_exclusions[
  spi_het_reg_results_wo_exclusions$term =='SAI_either:I(attribution_quintile %in% c(1, 2))TRUE'& spi_het_reg_results_wo_exclusions$Model == '3','estimate'] + 
  spi_het_reg_results_wo_exclusions[spi_het_reg_results_wo_exclusions$term =='SAI_either'& spi_het_reg_results_wo_exclusions$Model == '3','estimate']


z_stat = qnorm(0.975)

mod1 <- spi_het_reg_list_wo_exclusions[[1]]
recoded_term1 <- 'SAI_either:I(Affiliation %in% c("Very liberal", "Liberal"))TRUE'
base_term <- "SAI_either"

# compute the variance of the difference between base and recoded term
new_var1 <- get_the_variance_for_a_sum(mod1, base_term, recoded_term1)
new_se1 <- sqrt(new_var1)

# update the standard error and 95% confidence interval in the results data frame
idx1 <- spi_het_reg_results_wo_exclusions$Model == "1" & 
        spi_het_reg_results_wo_exclusions$term == recoded_term1

spi_het_reg_results_wo_exclusions[idx1, "std.error"] <- new_se1
spi_het_reg_results_wo_exclusions[idx1, "conf.low"] <- 
  spi_het_reg_results_wo_exclusions[idx1, "estimate"] - z_stat * new_se1
spi_het_reg_results_wo_exclusions[idx1, "conf.high"] <- 
  spi_het_reg_results_wo_exclusions[idx1, "estimate"] + z_stat * new_se1

# For Model 2
mod2 <- spi_het_reg_list_wo_exclusions[[2]]
recoded_term2 <- 'SAI_either:I(cc_concern_pre %in% c("Not concerned at all", "Slightly concerned"))TRUE'
base_term <- "SAI_either"

new_var2 <- get_the_variance_for_a_sum(mod2, base_term, recoded_term2)
new_se2 <- sqrt(new_var2)

idx2 <- spi_het_reg_results_wo_exclusions$Model == "2" & 
        spi_het_reg_results_wo_exclusions$term == recoded_term2

spi_het_reg_results_wo_exclusions[idx2, "std.error"] <- new_se2
spi_het_reg_results_wo_exclusions[idx2, "conf.low"] <- 
  spi_het_reg_results_wo_exclusions[idx2, "estimate"] - z_stat * new_se2
spi_het_reg_results_wo_exclusions[idx2, "conf.high"] <- 
  spi_het_reg_results_wo_exclusions[idx2, "estimate"] + z_stat * new_se2

# For Model 3
mod3 <- spi_het_reg_list_wo_exclusions[[3]]
recoded_term3 <- 'SAI_either:I(attribution_quintile %in% c(1, 2))TRUE'
base_term <- "SAI_either"

new_var3 <- get_the_variance_for_a_sum(mod3, base_term, recoded_term3)
new_se3 <- sqrt(new_var3)

idx3 <- spi_het_reg_results_wo_exclusions$Model == "3" & 
        spi_het_reg_results_wo_exclusions$term == recoded_term3

spi_het_reg_results_wo_exclusions[idx3, "std.error"] <- new_se3
spi_het_reg_results_wo_exclusions[idx3, "conf.low"] <- 
  spi_het_reg_results_wo_exclusions[idx3, "estimate"] - z_stat * new_se3
spi_het_reg_results_wo_exclusions[idx3, "conf.high"] <- 
  spi_het_reg_results_wo_exclusions[idx3, "estimate"] + z_stat * new_se3

df_het <- spi_het_reg_results_wo_exclusions |> 
  # Recode the term names
  mutate(term = case_when(
    term == "SAI_either" & Model =="1" ~ "Provided SAI info (not liberal)", 
    term == 'SAI_either:I(Affiliation %in% c("Very liberal", "Liberal"))TRUE' ~ "Provided SAI info (liberal)",
    term == "SAI_either" & Model =="2" ~ "Provided SAI info (more concerned)", 
    term == 'SAI_either:I(cc_concern_pre %in% c("Not concerned at all", "Slightly concerned"))TRUE' ~ "Provided SAI info (less concerned)",
    term == "SAI_either" & Model =="3" ~ "Provided SAI info\n(Attributes more to human action)",
    term == "SAI_either:I(attribution_quintile %in% c(1, 2))TRUE" ~ "Provided SAI info\n(Attributes less to human action)",
    TRUE ~ term
  )) |> 
  mutate(order = case_when(
    Model == "1" & term == "Provided SAI info (not liberal)" ~ 5.6,
    Model == "1" & term == "Provided SAI info (liberal)" ~ 5.2,
    Model == "2" & term == "Provided SAI info (more concerned)" ~ 3.6,
    Model == "2" & term == "Provided SAI info (less concerned)" ~ 3.2,
    Model == "3" & term == "Provided SAI info\n(Attributes more to human action)" ~ 1.6,
    Model == "3" & term == "Provided SAI info\n(Attributes less to human action)" ~ 1,
    TRUE ~ NA_real_
  )) 

plt_het <- ggplot(df_het, aes(x = estimate, y = order, color = Model, shape = Model)) +
  geom_vline(xintercept = 0, color = "gray50", linetype = 2) +
  geom_errorbarh(aes(xmin = conf.low, xmax = conf.high), height = 0.2) +
  geom_point(size = 3, fill = "white") +
  scale_color_manual(values = c("1" = "#3B9AB2", "2" = "#E1AF00", "3" = "#F98400")) +
  scale_y_continuous(breaks = df_het$order,
                     labels = df_het$term,
                     expand = expansion(mult = c(0.1, 0.1))) +
  labs(x = "Estimates with 95% Confidence Intervals",
       y = "") +
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black"))
```

```{r}
#| echo: false
#| fig-width: 6
#| fig-height: 4
#| label: fig-het-main
#| fig-cap: "Heterogeneity in effect of SAI information on support for emission reduction policies"

## FIGURE 3
plt_het
```


For respondents who self-identify (pre-treatment) as “Liberal” or “Very liberal,” we cannot reject the null hypothesis that providing SAI information has no differential effect (model 1). In fact, the point estimate suggests that SAI information might have a slightly negative effect for liberals and a slightly positive effect for others ($p$-value on the main effect: `r  round(summary(spi_het_reg_list_wo_exclusions[["By ideology"]])$coefficients['SAI_either', "Pr(>|t|)"], 3)`; $p$-value on the difference: `r round(summary(spi_het_reg_list_wo_exclusions[["By ideology"]])$coefficients['SAI_either:I(Affiliation %in% c("Very liberal", "Liberal"))TRUE', "Pr(>|t|)"], 3)`). This contrasts with @cherry2021does, who find that SAI information encourages support for a carbon tax among more egalitarian and communitarian respondents (many of whom likely self-identify as liberals[^libegal]) and not among others. 

[^libegal]: According to @cherry2021does, "those with egalitarian and communitarian views tend to see the market as a source of inequality and unjustness, and thus tend to readily accept descriptions of risk that imply that the market should be regulated." 

For respondents who say they are “Not concerned at all” or “Slightly concerned” about global warming (pre-treatment), SAI information appears to increase their average support for mitigation policy by about .12 SD, though the difference between this effect and the effect for other respondents is not statistically significant ($p$=`r  round(summary(spi_het_reg_list_wo_exclusions[["By concern"]])$coefficients['SAI_either:I(cc_concern_pre %in% c("Not concerned at all", "Slightly concerned"))TRUE', "Pr(>|t|)"], 3)`).

Finally, for respondents in the bottom two quintiles in attributing global warming to human activity (pre-treatment), <!--  are much less supportive of mitigation policy (model 3), -->  SAI information does not have a statistically significant effect on average support for mitigation policy, although the point estimate suggests that it slightly increases their average support. Again the difference between this effect and the effect for other respondents is not statistically significant ($p$ =`r  round(summary(spi_het_reg_list_wo_exclusions[["By attribution"]])$coefficients['SAI_either:I(attribution_quintile %in% c(1, 2))TRUE', "Pr(>|t|)"], 3)`).

Although none of the differences is statistically significant, the estimated effect of SAI information on support for emissions reductions is consistently more positive for the subgroup of respondents with lower average support for emissions reductions (non-liberals, those less concerned about climate change, those who attribute less of climate change to human action).[^fig-het-table]  This could reflect a ceiling effect: SAI information can only noticeably increase support for emissions reductions for people who are not already highly concerned about climate change and fully supportive of emissions reductions.

[^fig-het-table]: The regression results in Appendix @tbl-het-main document these differences in level of support across subgroups.

The Appendix reports exploratory, non-preregistered analysis on other heterogeneous effects of SAI information. We find suggestive evidence that SAI information increases support for mitigation policy among older respondents and non-college graduates, though the relevant interaction terms are significant only at the .1 level (@tbl-het-main-others). 


```{r main-mechanisms}
other_attitudes_reg_list <- function(this_data){
  # SWI: standardized worry index
  swi_SAI <- estimatr::lm_robust(standardized_worry_index ~ SAI_either + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  # STT: standardized tech threat
  stt_SAI <- estimatr::lm_robust(standardized_tech_threat ~ SAI_either + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  # STT: standardized mech belief
  smb_SAI <- estimatr::lm_robust(standardized_mech_belief ~ SAI_either + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  # STT: standardized techno fix
  stf_SAI <- estimatr::lm_robust(standardized_techno_fix ~ SAI_either + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  
  list("DV: Worry about climate change" = swi_SAI, "DV: Perceived techno threat" = stt_SAI, "DV: Belief in efficacy of emissions reductions" = smb_SAI, "DV: Belief in techno fix" = stf_SAI)
}

other_attitudes_table_from_reg_list <- function(reg_list){
  # adding a row to say: yes we have covariates
  rows <- tribble(~term, ~term, ~term, ~term, ~term, 
               "Covariates", "Yes", "Yes", "Yes", "Yes")
  attr(rows, "position") <- 5 # put it at the bottom

  modelsummary::modelsummary(reg_list, stars = T, gof_map = c("nobs", "r.squared", "adj.r.squared"), coef_map = c("(Intercept)" = "Intercept", "SAI_either" = "Provided SAI info"), add_rows = rows,
                           width = c(2,1,1,1,1))
}

other_attitudes_rl <- other_attitudes_reg_list(D)
```


```{r}

other_attitudes_rl_bind <- rbind(
  other_attitudes_rl[[1]] |> estimatr::tidy() |> 
    mutate(term = case_when(term == "SAI_either" ~ "DV: Worry about climate change",
                            TRUE ~ term)), 
  other_attitudes_rl[[2]] |> estimatr::tidy() |> 
    mutate(term = case_when(term == "SAI_either" ~ "DV: Perceived techno threat",
                            TRUE ~ term)), 
  other_attitudes_rl[[3]] |> estimatr::tidy() |> 
    mutate(term = case_when(term == "SAI_either" ~ "DV: Belief in efficacy of\nemissions reductions",
                            TRUE ~ term)), 
    other_attitudes_rl[[4]] |> estimatr::tidy() |> 
    mutate(term = case_when(term == "SAI_either" ~ "DV: Belief in techno fix",
                            TRUE ~ term))) |> 
  filter(stringr::str_detect(term, "DV:"))

# reorder factors in term
other_attitudes_rl_bind$term = factor(other_attitudes_rl_bind$term, 
                                      levels = rev(c(
                                        "DV: Worry about climate change",
                                        "DV: Perceived techno threat",
                                        "DV: Belief in efficacy of\nemissions reductions",
                                        "DV: Belief in techno fix"
                                      )))
  

plt_other_attitudes <- ggplot(other_attitudes_rl_bind, aes(x = estimate, y = term)) +
  geom_vline(xintercept = 0, color = "gray50", linetype = 2) +
  geom_errorbarh(aes(xmin = conf.low, xmax = conf.high), height = 0.2) +
  geom_point(size = 3, fill = "white") +
  scale_y_discrete(#breaks = other_attitudes_rl_bind$order,
                     #labels = other_attitudes_rl_bind$term,
                     expand = expansion(mult = c(0.1, 0.1))) +
  labs(x = "Estimates with 95% Confidence Intervals",
       y = "") +
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black"))
```

```{r}
#| echo: false
#| fig-width: 6
#| fig-height: 3
#| label: fig-sai-on-other-outcomes
#| fig-cap: "Effect of SAI information on other attitudes"

## FIGURE 4
plt_other_attitudes
```


### Effect of SAI information on other attitudes

@fig-sai-on-other-outcomes shows the estimated effect of providing SAI information <!-- (either alone or in conjunction with decarbonization information) --> on other post-treatment outcomes designed to tap into possible mechanisms by which SAI information could affect policy preferences. 


We estimate that exposure to SAI information (i) increases post-treatment worry about global warming (as measured by our index) by about `r round(other_attitudes_rl[["DV: Worry about climate change"]]$coefficients["SAI_either"], 3)` standard deviations, though we cannot reject the null ($p$ = `r round(summary(other_attitudes_rl[["DV: Worry about climate change"]])$coefficients["SAI_either", "Pr(>|t|)"], 3)`), (ii) increases respondents’ perception that without emissions reductions addressing global warming may require “deploying unproven technologies with harmful consequences” ("perceived techno threat" in @fig-sai-on-other-outcomes) by about `r round(other_attitudes_rl[["DV: Perceived techno threat"]]$coefficients["SAI_either"], 3)` standard deviations (p < .001), <!-- `r round(summary(other_attitudes_rl[["DV: Perceived techno threat"]])$coefficients["SAI_either", "Pr(>|t|)"], 3)`),--> and (iii) decreases respondents’ confidence that global warming can be addressed without “significantly reducing carbon dioxide emissions” ("belief in techno fix" in @fig-sai-on-other-outcomes) by about `r round(other_attitudes_rl[["DV: Belief in techno fix"]]$coefficients["SAI_either"], 3)` standard deviations (p < .001). <!-- =  `r round(summary(other_attitudes_rl[["DV: Belief in techno fix"]])$coefficients["SAI_either", "Pr(>|t|)"], 3)`).--> (The latter two effects are significant even after applying a Bonferroni correction as specified in our PAP.) Our SAI treatment thus appears to generally make respondents more pessimistic about climate change on average: slightly more worried about its effects (though this evidence is only suggestive), more concerned about measures that might be taken to address it, and more skeptical that the problem can be addressed without reducing emissions. We view this as evidence that balanced SAI information serves as an attitudinal "clarion call" for many respondents [@merk2016knowledge], even if average effects on emissions policy preferences are muted.     

```{r other4-funcs}
reg_list_other_outcomes4 <- function(this_data){
  swi_condition <- estimatr::lm_robust(standardized_worry_index ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  stt_condition <- estimatr::lm_robust(standardized_tech_threat ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  smb_condition <- estimatr::lm_robust(standardized_mech_belief ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  stf_condition <- estimatr::lm_robust(standardized_techno_fix ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = this_data)
  
  list("DV: Worry about climate change" = swi_condition, "DV: Perceived techno threat" = stt_condition, "DV: Belief in efficacy of emissions reductions" = smb_condition, "DV: Belief in techno fix" = stf_condition)
}

table_for_other_outcomes4 <- function(reg_list){
  rows <- tribble(~term, ~term, ~term, ~term, ~term, 
               "Covariates", "Yes", "Yes", "Yes", "Yes")
  attr(rows, "position") <- 9 # put it at the bottom

  modelsummary::modelsummary(reg_list, stars = T, gof_map = c("nobs", "r.squared", "adj.r.squared"), coef_map = c("(Intercept)" = "Intercept", "ConditionSAI" = "Provided SAI info only", "ConditionEmissions" = "Provided decarbonization info only", "ConditionBoth" = "Provided both"), add_rows = rows,
                           width = c(6,2,2,2,2))
}
reg_list_other4 <- reg_list_other_outcomes4(D)
```


```{r}

reg_list_other4_bind <- rbind(
  reg_list_other4[[1]] |> estimatr::tidy() |> 
    mutate(model =  "DV: Worry about climate change"),
  reg_list_other4[[2]] |> estimatr::tidy() |> 
    mutate(model = "DV: Perceived techno threat"),
  reg_list_other4[[3]] |> estimatr::tidy() |> 
    mutate(model =  "DV: Belief in efficacy of\nemissions reductions"),
    reg_list_other4[[4]] |> estimatr::tidy() |> 
    mutate(model = "DV: Belief in techno fix")) |> 
  filter(stringr::str_detect(term, "Condition")) |> 
  mutate(term = recode(term,
    "ConditionEmissions"   = "Provided decarbonization info only",
    "ConditionSAI"         = "Provided SAI info only",
    "ConditionBoth"        = "Provided both"
  ), 
    term = factor(term, levels = c(
      "Provided decarbonization info only",
      "Provided SAI info only",
      "Provided both")))



reg_list_other4_bind$model = factor(reg_list_other4_bind$model, 
                                      levels = rev(c(
                                        "DV: Worry about climate change",
                                        "DV: Perceived techno threat",
                                        "DV: Belief in efficacy of\nemissions reductions",
                                        "DV: Belief in techno fix"
                                      )))

# Plot the results using a horizontal error bar and custom y-axis ordering.
plt_other4 <- reg_list_other4_bind |> 
  ggplot(aes(x = estimate, xmin = conf.low, xmax = conf.high, y = model, col = term, pch = term)) + 
  geom_pointrange(position = position_dodge(width = -.35)) + 
  geom_vline(xintercept = 0, lty = 3) + 
  scale_y_discrete() +
  scale_color_manual(values = c("Provided decarbonization info only" = "#3B9AB2", 
                                "Provided SAI info only" = "#E1AF00", 
                                "Provided both" = "#F98400")) +
  guides(color = guide_legend(nrow = 2, byrow = TRUE))+
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black")) +
  labs(x = "Estimates with 95% Confidence Intervals", y = "", pch = "", col = "")

```

```{r}
#| echo: false
#| fig-width: 6.3
#| fig-height: 4.2
#| label: fig-other-outcomes-by-treatment-arm
#| fig-cap: "Effect of each information treatment on other attitudes"

## FIGURE 5
plt_other4
```

In @fig-other-outcomes-by-treatment-arm we report the effect of each distinct information treatment (relative to control) on the same attitudes.[^papexploratory] We highlight the most important findings.

Of the three treatments, only providing SAI information on its own significantly increases worry about climate change relative to control. Our SAI information in isolation may have especially worried respondents because it emphasizes that emissions reductions won’t immediately cool the planet and/or because putting sulfuric acid in the atmosphere sounds risky. SAI information appears to worry respondents less when it is paired with a decarbonization message, though we cannot reject the null that the two effects are the same ($p$=`r round(get_the_p_for_a_diff(reg_list_other4[["DV: Worry about climate change"]], "ConditionBoth", "ConditionSAI"), 3)`).

[^papexploratory]: These analyses are pre-registered, but in our PAP we stated that we would treat them as exploratory given the multiple comparisons being made. The regression results appear in Appendix @tbl-sai-on-other-outcomes.


```{r}
#| echo: false
both_vs_emissions_p <- get_the_p_for_a_diff(reg_list_other4[["DV: Belief in efficacy of emissions reductions"]], "ConditionBoth", "ConditionEmissions")
```

Although above we found that providing SAI information has null average effects on belief in the efficacy of emissions reductions, @fig-other-outcomes-by-treatment-arm shows that it matters whether decarbonization information is also included: the combined treatment made respondents believe in emissions reduction efficacy more (by .079 SDs), while the effect of SAI information on its own is negative ($p$ value on difference in the two effects: `r round(both_vs_emissions_p, 3)`).


The last set of estimates in @fig-other-outcomes-by-treatment-arm shows that the decarbonization treatment, like the SAI treatment, reduced confidence in the existence of a "techno fix" that would obviate the need for emissions reductions (compared to control). The effect of the SAI treatment was larger, however ($p$ =`r round(get_the_p_for_a_diff(reg_list_other4[["DV: Belief in techno fix"]], "ConditionSAI", "ConditionEmissions"), 3)`). This suggests that a balanced message about SAI information could be even more effective than a decarbonization message at convincing people of the necessity of emissions reductions.  


## Discussion 

Our study investigated how providing balanced information about SAI as a potential temporary response to climate change, both with and without a message about the importance of decarbonization, affects public support for decarbonization policies and related attitudes in the United States. We found no significant effects of SAI information on support for emissions mitigation policies on average, which contrasts both with previous research showing a moral hazard or crowding-out effect [@raimi2019framing;@campbell-ervai2017influence] and with previous research documenting a “clarion call” or crowding-in effect [@merk2016knowledge;@cherry2021does]. We did, however, find some evidence that SAI information increases support for mitigation among respondents more skeptical of climate change, and we found that information about SAI makes climate change appear to be a more serious problem, consistent with the “clarion call” hypothesis. Thus our findings suggest that broader awareness of SAI will not reduce, and may even increase, support for decarbonization policy. 
 
Our study also highlights the importance of balanced messaging that portrays SAI as a potentially risky complement to decarbonization rather than as a substitute. Our SAI treatment highlighted the technology's risks, which may explain why it increased worry about climate change while not eroding support for decarbonization. It also increased confidence that emissions reductions would mitigate future warming, but only when paired with a message emphasizing the importance of decarbonization. Future studies should further investigate this and other ways in which combinations of messages alter the public perception of climate change and responses to it.


Our study has important limitations, some of which future work can address.  As in most research in this area [e.g. @cherry2021does; @raimi2019framing; @fairbrother2016geoengineering], our outcome measures are simply stated positions in an online survey; following @merk2016knowledge and @merk2024presenting, future work could examine how similar information treatments affect “real stakes” outcomes such as willingness to donate or sign a petition. Moreover, as in other studies our treatment is just a short text passage and image. The effects of an actual program of SAI that alters the climate (and the discourse around climate change) could be altogether different. 




{{< pagebreak >}}

## References

::: {#refs}
:::

{{< pagebreak >}}

## Appendix



### Survey details (question wording and flow)

- "In general, how would you describe your political viewpoint?" (Very liberal, liberal, Moderate, Conservative, Very conservative)
- [Background information about climate change; see @fig-control-message]
- "The average temperature on earth has increased by almost 2 degrees Fahrenheit since the mid-1800s. What percentage of this increase do you think is caused by human activities?" (0-100 slider)
- "How concerned are you about global warming?" (Not concerned at all, Slightly concerned, Moderately concerned, Extremely concerned)
- "Do you agree with the following statement? 'I believe I can do something to address global warming.'" (Strongly disagree, Disagree, Neither agree nor disagree, Somewhat agree, Strongly agree)
- [Treatments: Control, SAI, Decarbonization, Both; see @fig-treatments]
- "Do you agree with the following statement? 'Thanks to technology, there is a good chance that we will be able to completely solve the problem of global warming in the next 50 years without significantly reducing carbon dioxide emissions.'" (Strongly disagree, Disagree, Neither agree nor disagree, Somewhat agree, Strongly agree)
- "Do you agree with the following statement? 'Humanity could limit future increases in the global average temperature by reducing emissions of carbon dioxide now.'" (Strongly disagree, Disagree, Neither agree nor disagree, Somewhat agree, Strongly agree)
- "Do you agree with the following statement? 'If we don't take action to reduce emissions now, then responding to global warming in the future is likely to require deploying unproven technologies with harmful consequences.'" (Strongly disagree, Disagree, Neither agree nor disagree, Somewhat agree, Strongly agree)
- "Below is a list of potential ways to address global warming. Which of these do you think some experts view as promising?" (Experts do not view as promising, Some experts view as promising)
    - "Extracting silicates from agricultural soil"
    - "Releasing sulfur dioxide into the atmosphere"
    - "Pumping dry ice into the earth's crust"
    - "Converting heating systems to use electricity instead of gas"
- "How much do you think global warming will affect you personally?" (Not at all, A little, Somewhat, A lot, A great deal)
- "How much do you think global warming will negatively affect future generations of people?" (Not at all, A little, Somewhat, A lot, A great deal)
- "Some people argue that the risks from global warming have been greatly exaggerated. Also, some researchers believe that the responses that come from surveys like this one are not reliable. To show skeptics that respondents to this survey do pay attention, could you please answer both 'Strongly disagree' and 'Neither agree nor disagree' below?" (Strongly disagree, Disagree, Neither agree nor disagree, Somewhat agree, Strongly agree)
- "Now we would like to know more about how you think we should address global warming. Do you agree or disagree with the following statements?" (Strongly disagree, Disagree, Neither agree nor disagree, Somewhat agree, Strongly agree)
    - "The US should sign an international treaty to reduce emissions from the US and other countries."
    - "Car manufacturers should be required to increase the fuel efficiency of their vehicles."
    - "The US should increase subsidies for renewable energy such as wind and solar power."
    - "The US should introduce a tax on fuel based on the amount of carbon dioxide emitted by that fuel."
    - "The US should build more nuclear power plants."
    - "The US should encourage individuals to use less energy in their homes and vehicles."
    - "The US should reduce the emissions of factories and power plants."
    - "The US should use tax money to encourage the development of technology that cheaply produces clean energy."

### Sample size determination {#sec-sample-size}

```{r}
#| echo: false
#| message: false
#| warning: false
#| fig-width: 6
#| fig-height: 4
#| label: fig-power
#| fig-cap: "Power curves for detecting effect sizes at different sample sizes"

results <- tibble(x = seq(0, .2, by = .001)) %>% 
  expand_grid(n = c(200, 1000, 1500, 2500),
              C = 0.7,
              groups = 4) %>% 
  mutate(se_n = sqrt(2 * groups * C / n),
         power = 1 - pnorm(qnorm(0.975) * se_n, mean = x, sd = se_n),
         sample = factor(n)) %>% 
  mutate(
    C_label      = paste0("Variance factor: ", C),
    groups_label = paste0("Number of groups: ", groups)
  )

color_vals    <- c("200" = "#3B9AB2", "1000" = "#E1AF00", "1500" = "#F98400", "2500" = "#A3A500")
linetype_vals <- c("200" = "solid",   "1000" = "dashed",   "1500" = "dotted",  "2500" = "twodash")

ggplot(results, aes(x = x, y = power, color = sample, linetype = sample)) +
  geom_vline(xintercept = 0.15, color = "gray50", linetype = 2) +
  geom_hline(yintercept = c(0.05, 0.8), color = "gray50", linetype = 3) +
  geom_line(size = 1) +
  scale_color_manual(name = "Sample size", values = color_vals) +
  scale_linetype_manual(name = "Sample size", values = linetype_vals) +
  expand_limits(y = 0) +
  labs(
    x   = "Effect size (in SDs of outcome)",
    y   = "Power"
  ) +
  theme(
    legend.position      = "top",
    legend.key           = element_blank(),
    panel.background     = element_blank(),
    panel.grid.major     = element_blank(),
    panel.grid.minor     = element_blank(),
    axis.line            = element_line(color = "black"),
    axis.text            = element_text(color = "black"),
    text                 = element_text(size = 11)
  )
```

The target sample of 2,500 was set by our survey budget in light of an a priori power analysis. @fig-power shows power curves for detecting effect sizes at different sample sizes. Using pilot‐study data (which suggested our covariates would explain roughly 30% of the outcome variance), we calculated that, in a four-group design and for a two-tailed test at significance level of 0.05, a sample of 2,500 would give us over 80% power to detect an effect size of 0.15 of a standard deviation in the outcome variable. 

### Compliance with human subjects guidance

We affirm that the experimental design is consistent with APSA's Principles and Guidance for Human Subjects Research. 

The experiment does not use deception and has no potential harms. 

At the beginning of the survey, all participants gave consent to the following: "This survey is part of an academic research project about climate change. It should take about 5 minutes to complete. We will ask questions about you and we will ask you some questions about climate change. Your responses will be kept confidential. The resulting data will be used for academic research. It will contain no identifying information about you such as location or IP address. We will retain no information about you beyond what you explicitly tell us in the survey. The data will be stored on secure servers and may eventually be made available on an online replication archive. If at any point you wish to withdraw consent and your data, you may do so by exiting the survey. Do you understand the above information and consent to taking part in the survey?"

Participants were paid \$1.25 each via the CloudResearch platform. The median respondent completion time was 4 minutes and 51 seconds; the average completion time was 5 minutes and 53 seconds. 


### Additional figures and tables 


![Explanation of climate change (provided to all respondents)](images/climate_change.png){#fig-control-message width="65%"}


{{< pagebreak >}} 


```{r}
#| echo: false
#| output: false
sample_proportions <- D |> 
  mutate(White = Race == "White",
         Black = Race == "Black or African American",
         Asian = Race %in% c("Chinese", "Filipino", "Japanese", "Asian Indian", "Vietnamese", "Korean"),
         `American Indian or Alaska Native` = Race == "American Indian or Alaska Native",
         `Other race` = Race %in% c("An ethnicity not listed here", "Other"),
         Male = Sex == "Male",
         Female = Sex == "Female",
         `Age 18-29` = Age >= 18 & Age <= 29,
         `Age 30-39` = Age >= 30 & Age <= 39,
         `Age 40-49` = Age >= 40 & Age <= 49,
         `Age 50-59` = Age >= 50 & Age <= 59,
         `Age 60-69` = Age >= 60 & Age <= 69,
         `Age 70+` = Age >= 70,
         `High school` = str_detect(Education, "^High school graduate"),
         `Some college` = str_detect(Education, "^Some college"),
         `Associates degree` = str_detect(Education, "^Associate degree"),
         `Bachelors degree` = str_detect(Education, "^Bachelor"),
         `Masters degree` = str_detect(Education, "^Master"),
         `Doctorate` = str_detect(Education, "^Doctorate"),
         `HH income < $10k` = `Household Income` == "Less than $10,000",
         `HH income $10k-$20k` = `Household Income` == "$10,000-$19,999",
         `HH income $20k-$30k` = `Household Income` == "$20,000-$29,999",
         `HH income $30k-$40k` = `Household Income` == "$30,000-$39,999",
         `HH income $40k-$50k` = `Household Income` == "$40,000-$49,999",
         `HH income $50k-$60k` = `Household Income` == "$50,000-$59,999",
         `HH income $60k-$70k` = `Household Income` == "$60,000-$69,999",
         `HH income $70k-$80k` = `Household Income` == "$70,000-$79,999",
         `HH income $80k-$90k` = `Household Income` == "$80,000-$89,999",
         `HH income $90k-$100k` = `Household Income` == "$90,000-$99,999",
         `HH income $100k-$125k` = `Household Income` == "$100,000-$124,999",
         `HH income $125k-$150k` = `Household Income` == "$125,000-$149,999",
         `HH income $150k-$175k` = `Household Income` == "$150,0000-$174,999",
         `HH income $175k-$200k` = `Household Income` == "$175,000-$199,999",
         `HH income $200k+` = `Household Income` %in% c("$200,000-$224,999",
                                                        "$225,000-$249,999",
                                                        "$250,000 or more"),
         Democrat = `Political Party` == "Democrat",
         Republican = `Political Party` == "Republican",
         Independent = `Political Party` == "Independent",
         `Other party` = `Political Party` == "Something else") |> 
  summarize(across(White:`Other party`, ~ mean(.x, na.rm = T))) |> 
  pivot_longer(cols = everything(), names_to = "Characteristic", values_to = "Sample")

population_proportions <- tribble(~Characteristic, ~Population,
                           "Age 18-29", .2, 
                           "Age 30-39", .18,
                           "Age 40-49", .16,
                           "Age 50-59", .16,
                           "Age 60-69", .16,
                           "Age 70+", .15,
                           "High school", .29,
                           "Some college", .16,
                           "Associates degree", .1,
                           "Bachelors degree", .22,
                           "Masters degree", .09,
                           "Doctorate", .02,
                           "HH income < $10k", .03, 
                           "HH income $10k-$20k", .04,
                           "HH income $20k-$30k", .05,
                           "HH income $30k-$40k", .06,
                           "HH income $40k-$50k", .06,
                           "HH income $50k-$60k", .06, 
                           "HH income $60k-$70k", .06,
                           "HH income $70k-$80k", .06,
                           "HH income $80k-$90k", .05,
                           "HH income $90k-$100k", .05,
                           "HH income $100k-$125k", .11,
                           "HH income $125k-$150k", .08,
                           "HH income $150k-$175k", .07,
                           "HH income $175k-$200k", .05,
                           "HH income $200k+", .16,
                           "White", .755,
                           "Black", .136,
                           "Asian", .063,
                           "American Indian or Alaska Native", .016,
                           "Other race",  .03, 
                           "Male", .496,
                           "Female", .504,
                           "Democrat", .28,
                           "Republican", .3,
                           "Independent", .41,
                           "Other party", .01)

population_proportions |> 
  left_join(sample_proportions) -> joined_proportions
```


```{r}
#| echo: false
#| fig-height: 6
#| fig-width: 5.5
#| fig-cap: "Comparison of analysis sample to US population. Data on US population from the US Census and (for party affiliation) a March 2024 Gallup poll"
#| label: fig-rep
joined_proportions |> 
  pivot_longer(cols = -Characteristic, names_to = "Type") |> 
  mutate(Characteristic = fct_inorder(Characteristic)) |> 
  ggplot(aes(x = value, y = fct_rev(Characteristic), col = Type)) + 
  geom_point(alpha = .7) + 
  geom_line(aes(group = Characteristic), show.legend = F) + 
  labs(x = "Proportion", y = "", col = "") + 
  scale_color_manual(values = c("Population" = "#3B9AB2", "Sample" = "#E1AF00")) +
  #theme_bw() + 
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black"))

```
 

 


```{r}
#| echo: false
#| message: false
#| fig-cap: "Manipulation check: proportion saying each technology is viewed as promising by some experts, separately by treatment group"
#| label: fig-manip-check
D |> 
  select(Condition, starts_with("manip_check")) |> 
  pivot_longer(-Condition, names_to = "Question", names_prefix = "manip_check_post_2_") |> 
  left_join(tibble(Question = c("1", "2", "3", "4"), Technology = c('Extracting silicates from\nagricultural soil', 'Releasing sulfur dioxide\ninto the atmosphere', "Pumping dry ice into\nthe earth's crust", 'Converting heating systems\nto use electricity\ninstead of gas')), by = "Question") |> 
  group_by(Condition, Technology) |> 
  summarize(pr_promising = mean(value == "Some experts view as promising")) |> 
  ggplot(aes(x = Technology, y = pr_promising, col = Condition)) + 
  geom_point() + 
  coord_flip() + 
  labs(y = "Proportion saying some experts\nview given technology as promising") + 
  scale_color_manual(values = c("Neither" = "#3B9AB2", "Emissions" = "#E1AF00", "SAI" = "#FF0000",
                                "Both" = "#D5D5D3")) +
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black"))
  
```


```{r}
#| echo: false
#| label: tbl-manip-check
#| tbl-cap: "Manipulation check regression results"

manip_1 <- estimatr::lm_robust(I(manip_check_post_2_1 == "Some experts view as promising") ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

manip_2 <- estimatr::lm_robust(I(manip_check_post_2_2 == "Some experts view as promising") ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

manip_3 <- estimatr::lm_robust(I(manip_check_post_2_3 == "Some experts view as promising") ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

manip_4 <- estimatr::lm_robust(I(manip_check_post_2_4 == "Some experts view as promising") ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

# adding a row to say: yes we have covariates
rows <- tribble(~term, ~term, ~term,  ~term, ~term,  
               "Covariates", "Yes", "Yes", "Yes", "Yes") 
attr(rows, "position") <- 9 # put it at the bottom

modelsummary::modelsummary(list("Extract silicates from agricultural soil" = manip_1, 
                                "Release sulfur dioxide into the atmosphere" = manip_2, 
                                "Pump dry ice into the earth's crust" = manip_3, 
                                "Convert heating systems to use electricity instead of gas" = manip_4), 
                           stars = T, 
                           gof_map = c("nobs", "r.squared", "adj.r.squared"),
                           coef_map = c("(Intercept)" = "Intercept",
                                        "ConditionSAI" = "SAI info only", 
                                        "ConditionEmissions" = "Emissions info only", 
                                        "ConditionBoth" = "SAI and emission info"), 
                           add_rows = rows,
                           # width and theme arguments passed to tinytable::tt
                           width = c(5,3,3,3,3)) -> xx

xx |> 
  tinytable::group_tt(j = list("DV: Stated technology seen as promising" = 2:5))
  
```



<!-- main effects in table format moved here -->

```{r}
#| echo: false
#| label: tbl-main-results
#| tbl-cap: "Effect of information treatments on support for emission reduction policies"

main_result_table_from_reg_list <- function(reg_list){
  # adding a row to say: yes we have covariates
  rows <- tribble(~term, ~term, ~term, ~term, 
               "Covariates", "Yes", "Yes", "Yes")
  attr(rows, "position") <- 13 # put it at the bottom

  modelsummary::modelsummary(reg_list, 
                           stars = T, 
                           gof_map = c("nobs", "r.squared", "adj.r.squared"), 
                           coef_map = c("(Intercept)" = "Intercept", 
                                        "SAI_either" = "Provided SAI info", 
                                        "Emissions_either" = "Provided decarbonization info", 
                                        "ConditionSAI" = "Provided SAI info only", 
                                        "ConditionEmissions" = "Provided decarbonization info only", 
                                        "ConditionBoth" = "Provided both SAI and decarbonization info"), 
                           add_rows = rows,
                           width = c(4,1,1,1)) |> 
  tinytable::group_tt(j = list("DV: Index of support for mitigation policy" = 2:4))
}

main_result_table_from_reg_list(spi_reg_list_wo_exclusions)

```

To show how the misalignment of our sample with the US population might affect our results, we re-estimate our main results using a sample weighted to align with population margins in terms of age, education, income, race, sex, and political party. We use raking to generate the weights and run survey-weighted regressions [@lumley2017fitting].[^weighted_reg_no_pap] The results (@tbl-main-results-weighted) are qualitatively similar to the unweighted results, but we noe that the point estimate of the effect of the SAI message is larger. 



```{r}
#| echo: false
#| label: tbl-main-results-weighted
#| tbl-cap: "Effect of information treatments on support for emission reduction policies with sample weighted to align with population characteristics"

library(survey)

D_clean <- D |>
  mutate(
    # Recode Age groups
    Age_group = case_when(
      Age >= 18 & Age <= 29 ~ "Age 18-29",
      Age >= 30 & Age <= 39 ~ "Age 30-39",
      Age >= 40 & Age <= 49 ~ "Age 40-49",
      Age >= 50 & Age <= 59 ~ "Age 50-59",
      Age >= 60 & Age <= 69 ~ "Age 60-69",
      Age >= 70           ~ "Age 70+",
      TRUE ~ NA_character_
    ),
    # Recode Education
    Education_recode = case_when(
      str_detect(Education, "^High school graduate") ~ "High school",
      str_detect(Education, "^Some college") ~ "Some college",
      str_detect(Education, "^Associate degree") ~ "Associates degree",
      str_detect(Education, "^Bachelor") ~ "Bachelors degree",
      str_detect(Education, "^Master") ~ "Masters degree",
      str_detect(Education, "^Doctorate") ~ "Doctorate",
      TRUE ~ NA_character_
    ),
    # Recode Household Income
    Income_group = case_when(
      `Household Income` == "Less than $10,000" ~ "HH income < $10k",
      `Household Income` == "$10,000-$19,999" ~ "HH income $10k-$20k",
      `Household Income` == "$20,000-$29,999" ~ "HH income $20k-$30k",
      `Household Income` == "$30,000-$39,999" ~ "HH income $30k-$40k",
      `Household Income` == "$40,000-$49,999" ~ "HH income $40k-$50k",
      `Household Income` == "$50,000-$59,999" ~ "HH income $50k-$60k",
      `Household Income` == "$60,000-$69,999" ~ "HH income $60k-$70k",
      `Household Income` == "$70,000-$79,999" ~ "HH income $70k-$80k",
      `Household Income` == "$80,000-$89,999" ~ "HH income $80k-$90k",
      `Household Income` == "$90,000-$99,999" ~ "HH income $90k-$100k",
      `Household Income` == "$100,000-$124,999" ~ "HH income $100k-$125k",
      `Household Income` == "$125,000-$149,999" ~ "HH income $125k-$150k",
      `Household Income` == "$150,0000-$174,999" ~ "HH income $150k-$175k",
      `Household Income` == "$175,000-$199,999" ~ "HH income $175k-$200k",
      `Household Income` %in% c("$200,000-$224,999", "$225,000-$249,999", "$250,000 or more") ~ "HH income $200k+",
      TRUE ~ NA_character_
    ),
    # Recode Race
    Race_recode = case_when(
      Race == "White" ~ "White",
      Race == "Black or African American" ~ "Black",
      Race %in% c("Chinese", "Filipino", "Japanese", "Asian Indian", "Vietnamese", "Korean") ~ "Asian",
      Race == "American Indian or Alaska Native" ~ "American Indian or Alaska Native",
      Race %in% c("An ethnicity not listed here", "Other") ~ "Other race",
      TRUE ~ NA_character_
    ),
    # Recode Sex
    Sex_recode = case_when(
      Sex == "Male" ~ "Male",
      Sex == "Female" ~ "Female",
      TRUE ~ NA_character_
    ),
    # Recode Political Party
    Political_Party_recode = case_when(
      `Political Party` == "Democrat" ~ "Democrat",
      `Political Party` == "Republican" ~ "Republican",
      `Political Party` == "Independent" ~ "Independent",
      `Political Party` == "Something else" ~ "Other party",
      TRUE ~ NA_character_
    )
  )

D_clean_complete <- D_clean |>
  filter(!is.na(Age_group),
         !is.na(Education_recode),
         !is.na(Income_group),
         !is.na(Race_recode),
         !is.na(Sex_recode),
         !is.na(Political_Party_recode))

dsgn <- svydesign(ids = ~1, data = D_clean_complete, weights = ~1)

pop_age_df <- population_proportions |> 
  filter(grepl("^Age", Characteristic)) |> 
  select(Characteristic, Population) |> 
  rename(Age_group = Characteristic, Freq = Population)

pop_edu_df <- population_proportions |> 
  filter(Characteristic %in% c("High school", "Some college", "Associates degree",
                                "Bachelors degree", "Masters degree", "Doctorate")) |> 
  select(Characteristic, Population) |> 
  rename(Education_recode = Characteristic, Freq = Population)

pop_income_df <- population_proportions |> 
  filter(grepl("^HH income", Characteristic)) |> 
  select(Characteristic, Population) |> 
  rename(Income_group = Characteristic, Freq = Population)

pop_race_df <- population_proportions |> 
  filter(Characteristic %in% c("White", "Black", "Asian", 
                               "American Indian or Alaska Native", "Other race")) |> 
  select(Characteristic, Population) |> 
  rename(Race_recode = Characteristic, Freq = Population)

pop_sex_df <- population_proportions |> 
  filter(Characteristic %in% c("Male", "Female")) |> 
  select(Characteristic, Population) |> 
  rename(Sex_recode = Characteristic, Freq = Population)

pop_party_df <- population_proportions |> 
  filter(Characteristic %in% c("Democrat", "Republican", "Independent", "Other party")) |> 
  select(Characteristic, Population) |> 
  rename(Political_Party_recode = Characteristic, Freq = Population)

# rake the survey design
raked_dsgn <- rake(design = dsgn,
                   sample.margins = list(~Age_group, ~Education_recode, ~Income_group, ~Race_recode, ~Sex_recode, ~Political_Party_recode),
                   population.margins = list(pop_age_df, pop_edu_df, pop_income_df, pop_race_df, pop_sex_df, pop_party_df))



spi_wtd_reg_list_given_data <- function(this_data){
    spi_SAI <- svyglm(standardized_policy_index ~ SAI_either + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, design = this_data)
    
    spi_Emissions <- svyglm(standardized_policy_index ~ Emissions_either + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, design = this_data)
    
    spi_all_four <- svyglm(standardized_policy_index ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, design = this_data)
    
    list(spi_SAI, spi_Emissions, spi_all_four)
    
}

spi_wtd_reg_list_wo_exclusions <- spi_wtd_reg_list_given_data(raked_dsgn)

main_result_table_from_wtd_reg_list <- function(wtd_reg_list){
  # adding a row to say: yes we have covariates
  rows <- tribble(~term, ~term, ~term, ~term, 
               "Covariates", "Yes", "Yes", "Yes")
  attr(rows, "position") <- 13 # put it at the bottom

  modelsummary::modelsummary(wtd_reg_list, 
                           stars = T, 
                           gof_map = c("nobs", "r.squared", "adj.r.squared"), 
                           coef_map = c("(Intercept)" = "Intercept", 
                                        "SAI_either" = "Provided SAI info", 
                                        "Emissions_either" = "Provided decarbonization info", 
                                        "ConditionSAI" = "Provided SAI info only", 
                                        "ConditionEmissions" = "Provided decarbonization info only", 
                                        "ConditionBoth" = "Provided both SAI and decarbonization info"), 
                           add_rows = rows,
                           width = c(4,1,1,1)) |> 
  tinytable::group_tt(j = list("DV: Index of support for mitigation policy" = 2:4))
}

main_result_table_from_reg_list(spi_wtd_reg_list_wo_exclusions)

```



```{r}
#| label: fig-sai-on-individual-outcomes
#| fig-cap: "Average effect of providing SAI information on respondents' support for US taking individual policy steps"

x <- estimatr::lm_robust(I(as.integer(response_treaty)) ~ SAI_either + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

mod_list <- list("Sign international treaty to reduce emissions" = x,
                 "Require higher fuel efficiency" = update(x, I(as.integer(response_car_eff)) ~ .),
                 "Increase subsidies for renewable energy" = update(x, I(as.integer(response_renewables)) ~ .),
                 "Introduce a carbon tax" = update(x, I(as.integer(response_carbon_tax)) ~ .),
                 "Build more nuclear power plants" = update(x, I(as.integer(response_nuclear)) ~ .),
                 "Encourage individuals to use less energy" = update(x, I(as.integer(response_ind_use)) ~ .),
                 "Reduce emissions of factories and power plants" = update(x, I(as.integer(response_elec_emit)) ~ .),
                 "Use tax money to develop clean energy" = update(x, I(as.integer(response_clean_nrg)) ~ .))

tibble(name = names(mod_list),
       mod = mod_list) |> 
  mutate(tidied = map(mod, broom::tidy)) |> 
  unnest(tidied) |> 
  filter(term == "SAI_either") |> 
  mutate(name = fct_inorder(name)) |> 
  ggplot(aes(x = estimate, xmin = conf.low, xmax = conf.high, y = fct_rev(name))) + 
  geom_pointrange() + 
  geom_vline(xintercept = 0, lty = 3) + 
  labs(x = "Estimates with 95% Confidence Intervals", y = "")+
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black"))
```



```{r}
#| label: fig-info-on-individual-outcomes
#| fig-cap: "Average effect of each information treatment on respondents' support for US taking individual policy steps"

x <- estimatr::lm_robust(I(as.integer(response_treaty)) ~ Condition + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

mod_list <- list("Sign international treaty to reduce emissions" = x,
                 "Require higher fuel efficiency" = update(x, I(as.integer(response_car_eff)) ~ .),
                 "Increase subsidies for renewable energy" = update(x, I(as.integer(response_renewables)) ~ .),
                 "Introduce a carbon tax" = update(x, I(as.integer(response_carbon_tax)) ~ .),
                 "Build more nuclear power plants" = update(x, I(as.integer(response_nuclear)) ~ .),
                 "Encourage individuals to use less energy" = update(x, I(as.integer(response_ind_use)) ~ .),
                 "Reduce emissions of factories and power plants" = update(x, I(as.integer(response_elec_emit)) ~ .),
                 "Use tax money to develop clean energy" = update(x, I(as.integer(response_clean_nrg)) ~ .))

tibble(name = names(mod_list),
       mod = mod_list) |> 
  mutate(tidied = map(mod, broom::tidy)) |> 
  unnest(tidied) |> 
  filter(str_detect(term, "Condition")) |> 
  mutate(name = fct_inorder(name),
         term = str_remove(term, "Condition"),
         term = factor(term, levels = c("Emissions", "SAI", "Both"))) |> 
  ggplot(aes(x = estimate, xmin = conf.low, xmax = conf.high, y = fct_rev(name), col = term, pch = term)) + 
  geom_pointrange(position = position_dodge(width = -.35)) + 
  geom_vline(xintercept = 0, lty = 3) + 
  scale_color_manual(values = c("Emissions" = "#3B9AB2", "SAI" = "#E1AF00", "Both" = "#F98400")) +
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black")) +
  labs(x = "Estimates with 95% Confidence Intervals", y = "", pch = "", col = "")
```

<!-- heterogeneity results in table format -->

```{r}
#| echo: false
#| label: tbl-het-main
#| tbl-cap: "Heterogeneity in effect of SAI information on support for emission reduction policies"


het_reg_table_given_reg_list(spi_het_reg_list_wo_exclusions)

```


```{r main-mechanisms-table}
#| echo: false
#| label: tbl-sai-on-other-outcomes
#| tbl-cap: "Effect of SAI information on other attitudes"


other_attitudes_table_from_reg_list(other_attitudes_rl)
```



```{r}
#| echo: false
#| label: tbl-other-outcomes-by-treatment-arm
#| tbl-cap: "Effect of each information treatment on other attitudes"


table_for_other_outcomes4(reg_list_other4)
```



```{r main-effect-heterogeneity-extra}
#| echo: false
D |> 
  mutate(mod = Affiliation == "Moderate",
         young = Age %in% 18:49,
         college = Education %in% c("Bachelor's degree (for example: BA, AB, BS)", "Doctorate degree (for example: PhD, EdD)", "Master's degree (for example: MA, MS, MEng, MEd, MSW, MBA)", "Professional degree (for example: MD, DDS, DVM, LLB, JD)"),
         poorer = `Household Income` %in% c("$10,000-$19,999", "$20,000-$29,999", "$30,000-$39,999", "$40,000-$49,999", "$50,000-$59,999", "$60,000-$69,999", "$70,000-$79,999")) -> Dpe2
         

spi_SAI_by_age <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*young + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = Dpe2)
spi_SAI_by_college <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*college + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = Dpe2)
spi_SAI_by_income <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*poorer + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = Dpe2)

spi_SAI_by_mod <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*mod + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = Dpe2)
spi_SAI_by_mod_att <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*I(attribution_quintile == 3) + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = Dpe2)
spi_SAI_by_mod_worry <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*I(cc_concern_pre == "Moderately concerned") + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = Dpe2)


```


```{r}
#| echo: false
#| label: tbl-het-main-others
#| tbl-cap: "Heterogeneity in effect of SAI information on support for emission reduction policies by age, education, and income (exploratory analysis)"

rows <- tribble(~term, ~term, ~term, ~term,
               "Covariates", "Yes", "Yes", "Yes")
attr(rows, "position") <- 17 # put it at the bottom

modelsummary::modelsummary(list("By age" = spi_SAI_by_age, "By education" = spi_SAI_by_college, "By income" = spi_SAI_by_income), 
                           stars = T, 
                           gof_map = c("nobs", "r.squared", "adj.r.squared"), 
                           coef_map = c("(Intercept)" = "Intercept", 
                                        "SAI_either" = "Provided SAI info", 
                                        'modTRUE' = "Moderate",
                                        'SAI_either:modTRUE' = "SAI info X Moderate",
                                        'youngTRUE' = "Age below 50", 
                                        'SAI_either:youngTRUE' = "SAI info X Age below 50",
                                        "collegeTRUE" = "College graduate", 
                                        "SAI_either:collegeTRUE" = "SAI info X College graduate",
                                        "poorerTRUE" = "HH income < 80k", 
                                        "SAI_either:poorerTRUE" = "SAI info X HH income < 80k"), 
                           add_rows = rows) |>
  tinytable::group_tt(j = list("DV: Index of support for mitigation policy" = 2:4))

```




```{r}
spi_SAI_by_affiliation_full <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

spi_SAI_by_concern_full <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*cc_concern_pre + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

spi_SAI_by_attribution_full <- estimatr::lm_robust(standardized_policy_index ~ SAI_either*attribution_quintile + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

# what could I show? effect at different levels. levels at different levels (by treatment group)
all_levels <- expand_grid(attribution_quintile = factor(unique(D$attribution_quintile), levels = levels(D$attribution_quintile)),
            Affiliation = factor(unique(D$Affiliation), levels = levels(D$Affiliation)),
            cc_concern_pre = factor(unique(D$cc_concern_pre), levels = levels(D$cc_concern_pre)),
            self_efficacy_pre = factor(unique(D$self_efficacy_pre), levels = levels(D$self_efficacy_pre)),
            SAI_either = c(0,1))
```


```{r preds-by-attribution}
#| label: fig-preds-by-attribution
#| fig-cap: "Predicted support for mitigation policy by treatment status and pre-treatment attribution of climate change to human activity (other predictors set at midpoint)"

preds_by_attribution <- predict(spi_SAI_by_attribution_full, 
        newdata = all_levels,
        interval = "confidence")

all_levels |> 
  mutate(fit = preds_by_attribution$fit[,"fit"],
         lwr = preds_by_attribution$fit[,"lwr"],
         upr = preds_by_attribution$fit[,"upr"]) -> all_levels_augmented

all_levels_augmented |> 
  filter(Affiliation == "Moderate",
         cc_concern_pre == "Moderately concerned",
         self_efficacy_pre == "Neither agree nor disagree") |> 
  mutate(attribution_quintile = factor(attribution_quintile, levels = 1:5)) |> 
  ggplot(aes(x = attribution_quintile, y = fit, ymax = upr, ymin = lwr, col = factor(SAI_either), shape = factor(SAI_either))) + 
    scale_color_manual(values = c("0" = "#3B9AB2", "1" = "#E1AF00")) +
  geom_pointrange(position = position_dodge(width = .25)) + 
  labs(x = "Degree to which respondent attributes\nclimate change to human activities (quintiles)",
       y = "Predicted support for mitigation policy\n(standardized index)",
       col = "SAI information \nprovided",
       shape = "SAI information \nprovided") + 
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black"))
```

```{r preds-by-affiliation}
#| label: fig-preds-by-affiliation
#| fig-cap: "Predicted support for mitigation policy by treatment status and pre-treatment political affiliation (other predictors set at midpoint)"

preds_by_affiliation <- predict(spi_SAI_by_affiliation_full, 
        newdata = all_levels,
        interval = "confidence")

all_levels |> 
  mutate(fit = preds_by_affiliation$fit[,"fit"],
         lwr = preds_by_affiliation$fit[,"lwr"],
         upr = preds_by_affiliation$fit[,"upr"]) -> all_levels_augmented

all_levels_augmented |> 
  filter(attribution_quintile == 3,
         cc_concern_pre == "Moderately concerned",
         self_efficacy_pre == "Neither agree nor disagree") |> 
  mutate(Affiliation = factor(Affiliation, levels = c("Very conservative", "Conservative", "Moderate", "Liberal", "Very liberal"))) |> 
  ggplot(aes(x = Affiliation, y = fit, ymax = upr, ymin = lwr, col = factor(SAI_either), shape = factor(SAI_either))) + 
  geom_pointrange(position = position_dodge(width = .25)) + 
    scale_color_manual(values = c("0" = "#3B9AB2", "1" = "#E1AF00")) +
  labs(x = "Respondent's self-reported political affiliation",
       y = "Predicted support for mitigation policy\n(standardized index)",
       col = "SAI information\nprovided",
       shape = "SAI information\nprovided") + 
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black")) +
  theme(axis.text.x = element_text(angle = 30, vjust = 1, hjust=1)) 
```

```{r preds-by-concern}
#| label: fig-preds-by-cc-concern
#| fig-cap: "Predicted support for mitigation policy by treatment status and pre-treatment concern about climate change (other predictors set at midpoint)"

preds_by_concern <- predict(spi_SAI_by_concern_full, 
        newdata = all_levels,
        interval = "confidence")

all_levels |> 
  mutate(fit = preds_by_concern$fit[,"fit"],
         lwr = preds_by_concern$fit[,"lwr"],
         upr = preds_by_concern$fit[,"upr"]) -> all_levels_augmented

all_levels_augmented |> 
  filter(attribution_quintile == 3,
         Affiliation == "Moderate",
         self_efficacy_pre == "Neither agree nor disagree") |> 
  mutate(cc_concern_pre = factor(cc_concern_pre, levels = c("Not concerned at all", "Slightly concerned", "Moderately concerned", "Very concerned", "Extremely concerned"))) |> 
  ggplot(aes(x = cc_concern_pre, y = fit, ymax = upr, ymin = lwr, col = factor(SAI_either), shape = factor(SAI_either))) + 
  geom_pointrange(position = position_dodge(width = .25)) + 
      scale_color_manual(values = c("0" = "#3B9AB2", "1" = "#E1AF00")) +
  labs(x = "Respondent's concern about climate change",
       y = "Predicted support for mitigation policy\n(standardized index)",
       col = "SAI information\nprovided",
       shape = "SAI information\nprovided") + 
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black")) + 
  theme(axis.text.x = element_text(angle = 30, vjust = 1, hjust=1)) 
```


```{r}
spi_SAI_by_affiliation_full <- estimatr::lm_robust(standardized_worry_index ~ SAI_either*Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

spi_SAI_by_concern_full <- estimatr::lm_robust(standardized_worry_index ~ SAI_either*cc_concern_pre + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

spi_SAI_by_attribution_full <- estimatr::lm_robust(standardized_worry_index ~ SAI_either*attribution_quintile + Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D)

# what could I show? effect at different levels. levels at different levels (by treatment group)
all_levels <- expand_grid(attribution_quintile = factor(unique(D$attribution_quintile), levels = levels(D$attribution_quintile)),
            Affiliation = factor(unique(D$Affiliation), levels = levels(D$Affiliation)),
            cc_concern_pre = factor(unique(D$cc_concern_pre), levels = levels(D$cc_concern_pre)),
            self_efficacy_pre = factor(unique(D$self_efficacy_pre), levels = levels(D$self_efficacy_pre)),
            SAI_either = c(0,1))
```


```{r preds-by-attribution-worry}
#| label: fig-preds-by-attribution-worry
#| fig-cap: "Predicted concern about climate change by treatment status and pre-treatment attribution of climate change to human activity (other predictors set at midpoint)"

preds_by_attribution <- predict(spi_SAI_by_attribution_full, 
        newdata = all_levels,
        interval = "confidence")

all_levels |> 
  mutate(fit = preds_by_attribution$fit[,"fit"],
         lwr = preds_by_attribution$fit[,"lwr"],
         upr = preds_by_attribution$fit[,"upr"]) -> all_levels_augmented

all_levels_augmented |> 
  filter(Affiliation == "Moderate",
         cc_concern_pre == "Moderately concerned",
         self_efficacy_pre == "Neither agree nor disagree") |> 
  mutate(attribution_quintile = factor(attribution_quintile, levels = 1:5)) |> 
  ggplot(aes(x = attribution_quintile, y = fit, ymax = upr, ymin = lwr, col = factor(SAI_either), shape = factor(SAI_either))) + 
    scale_color_manual(values = c("0" = "#3B9AB2", "1" = "#E1AF00")) +
  geom_pointrange(position = position_dodge(width = .25)) + 
  labs(x = "Degree to which respondent attributes\nclimate change to human activities (quintiles)",
       y = "Predicted concern about climate change\n(standardized index)",
       col = "SAI information\nprovided",
       shape = "SAI information\nprovided") + 
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black"))
```

```{r preds-by-affiliation-worry}
#| label: fig-preds-by-affiliation-worry
#| fig-cap: "Predicted concern about climate change by treatment status and pre-treatment political affiliation (other predictors set at midpoint)"

preds_by_affiliation <- predict(spi_SAI_by_affiliation_full, 
        newdata = all_levels,
        interval = "confidence")

all_levels |> 
  mutate(fit = preds_by_affiliation$fit[,"fit"],
         lwr = preds_by_affiliation$fit[,"lwr"],
         upr = preds_by_affiliation$fit[,"upr"]) -> all_levels_augmented

all_levels_augmented |> 
  filter(attribution_quintile == 3,
         cc_concern_pre == "Moderately concerned",
         self_efficacy_pre == "Neither agree nor disagree") |> 
  mutate(Affiliation = factor(Affiliation, levels = c("Very conservative", "Conservative", "Moderate", "Liberal", "Very liberal"))) |> 
  ggplot(aes(x = Affiliation, y = fit, ymax = upr, ymin = lwr, col = factor(SAI_either), shape = factor(SAI_either))) + 
  geom_pointrange(position = position_dodge(width = .25)) + 
  scale_color_manual(values = c("0" = "#3B9AB2", "1" = "#E1AF00")) +
  labs(x = "Respondent's self-reported political affiliation",
       y = "Predicted concern about climate change\n(standardized index)",
       col = "SAI information\nprovided",
       shape = "SAI information\nprovided") + 
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black")) + 
  theme(axis.text.x = element_text(angle = 30, vjust = 1, hjust=1)) 
```

```{r preds-by-concern-worry}
#| label: fig-preds-by-cc-concern-worry
#| fig-cap: "Predicted concern about climate change by treatment status and pre-treatment concern about climate change (other predictors set at midpoint)"

preds_by_concern <- predict(spi_SAI_by_concern_full, 
        newdata = all_levels,
        interval = "confidence")

all_levels |> 
  mutate(fit = preds_by_concern$fit[,"fit"],
         lwr = preds_by_concern$fit[,"lwr"],
         upr = preds_by_concern$fit[,"upr"]) -> all_levels_augmented

all_levels_augmented |> 
  filter(attribution_quintile == 3,
         Affiliation == "Moderate",
         self_efficacy_pre == "Neither agree nor disagree") |> 
  mutate(cc_concern_pre = factor(cc_concern_pre, levels = c("Not concerned at all", "Slightly concerned", "Moderately concerned", "Very concerned", "Extremely concerned"))) |> 
  ggplot(aes(x = cc_concern_pre, y = fit, ymax = upr, ymin = lwr, col = factor(SAI_either), shape = factor(SAI_either))) + 
  geom_pointrange(position = position_dodge(width = .25)) + 
  scale_color_manual(values = c("0" = "#3B9AB2", "1" = "#E1AF00")) +
  labs(x = "Respondent's concern about climate change",
       y = "Predicted concern about climate change\n(standardized index)",
       col = "SAI information\nprovided",
       shape = "SAI information\nprovided") + 
  theme(text = element_text(size = 10.5),
        legend.position = "top",
        legend.key = element_blank(),
        panel.grid.major = element_blank(),
        axis.text.x = element_text(size = 10.5, colour = "black"),
        axis.text.y = element_text(colour = "black"),
        plot.caption = element_text(size = 10.5, hjust = -.02),
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        axis.line = element_line(colour = "black")) + 
  theme(axis.text.x = element_text(angle = 30, vjust = 1, hjust=1)) 
```


\clearpage

{{< pagebreak >}}

### Exclusions from the sample

Our pre-analysis plan stated that we would exclude respondents who took too little time or too much time to complete the survey, or failed an attention check. Our main analysis deviates from this plan, showing results for the full sample instead. Here we report results following the pre-registered exclusion plan.

Specifically, we exclude respondents with log response times outside the interval given by 
$$ [q_{25} - 1.5 \text{IQR}, q_{75} + 1.5 \text{IQR}],$$
where $q_x$ is the $x$th percentile of log response times and IQR is the interquartile range of log response times. (Because treatment status could have affected response times, both the percentiles and the interquartile range were computed by treatment condition.)

We also exclude respondents who failed a (post-treatment) attention check in which we asked respondents to select two options on a Likert scale of agreement. We found that 31 respondents entered a single response; 108 entered two responses that were not the ones we requested. Following our pre-analysis plan, we check whether failure of the attention check was predicted by treatment status. We were unable to reject the null hypothesis of no relationship (p=.110), so we proceeded with excluding respondents who failed the attention check. 

Combined, the response time filtering and attention check removes 171 respondents from the survey so that, after exclusions, we are left with a sample size of n = 2,338.

Again following the pre-analysis plan, we used an F-test to check for balance in our prognostic covariates between each treatment group and the control group in the post-exclusion data. (We stated that if we detected imbalance that would be substantially improved by including the excluded respondents, we would use the full pre-exclusion dataset.) As shown in the first three columns of @tbl-post-exclusion-balance in the Appendix, the $p$-value in each case is above .01 (the threshold specified in our pre-analysis plan), which means we could proceed with excluding respondents who failed the attention check. 

```{r}
#| echo: false
#| label: tbl-post-exclusion-balance
#| tbl-cap: "Omnibus balance tests (relative to control)"
#| 
balance_reg_Emissions <- estimatr::lm_robust(I(Condition == "Emissions") ~ Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D_post_exclusions |> filter(Condition %in% c("Neither", "Emissions"))) 

balance_reg_SAI <- estimatr::lm_robust(I(Condition == "SAI") ~ Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D_post_exclusions |> filter(Condition %in% c("Neither", "SAI"))) 

balance_reg_Both <- estimatr::lm_robust(I(Condition == "Both") ~ Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D_post_exclusions |> filter(Condition %in% c("Neither", "Both"))) 

balance_reg_Emissions_full <- estimatr::lm_robust(I(Condition == "Emissions") ~ Affiliation + cc_concern_pre + attribution_quintile + self_efficacy_pre, data = D |> filter(Condition %in% c("Neither", "Emissions"))) 


fstat_p_val_from_reg <- function(reg){
  fstat <- summary(reg)$fstatistic
  1 - pf(fstat[1], fstat[2], fstat[3])
}

# adding a row at the bottom with the $p$-values on the F-tests
rows <- tribble(~term, ~term, ~term, ~term, ~term,
                # see function for getting F-test $p$-value from regression output 
               "$p$-value from F-test", fstat_p_val_from_reg(balance_reg_Emissions), fstat_p_val_from_reg(balance_reg_SAI), fstat_p_val_from_reg(balance_reg_Both), fstat_p_val_from_reg(balance_reg_Emissions_full))
attr(rows, "position") <- 37 # put it at the bottom

modelsummary::modelsummary(list("Emissions" = balance_reg_Emissions, "SAI" = balance_reg_SAI, "Both" = balance_reg_Both, "Emissions w/ exclusions" = balance_reg_Emissions_full), stars = T, gof_map = c("nobs"), add_rows = rows,
                           width = c(7,1.75,1.75,1.75,2),
                           coef_map = c("(Intercept)" = "Intercept",
                                        "AffiliationVery conservative" = "Very conservative",
                                        "AffiliationConservative" = "Conservative",
                                        "AffiliationLiberal" = "Liberal",
                                        "AffiliationVery liberal" = "Very liberal",
                                        "cc_concern_preNot concerned at all" = "Not concerned at all about CC",
                                        "cc_concern_preSlightly concerned" = "Slightly concerned about CC",
                                        "cc_concern_preVery concerned" = "Very concerned about CC",
                                        "cc_concern_preExtremely concerned" = "Extremely concerned about CC",
                                        "attribution_quintile1" = "Humans responsible for CC: Lowest",
                                        "attribution_quintile2" = "Humans responsible for CC: Med low",
                                        "attribution_quintile4" = "Humans responsible for CC: Med high",
                                        "attribution_quintile5" = "Humans responsible for CC: High",
                                        "self_efficacy_preStrongly disagree" = "Can do something about CC: Strongly disagree",
                                        "self_efficacy_preSomewhat disagree" = "Can do something about CC: Somewhat disagree",
                                        "self_efficacy_preSomewhat agree" = "Can do something about CC: Somewhat agree",
                                        "self_efficacy_preStrongly agree" = "Can do something about CC: Strongly agree")) |>  
  tinytable::group_tt(j = list("DV: Treated" = 2:5)) |> 
  tinytable::style_tt(fontsize = .75, 
                      bootstrap_css = "padding-right: .05em; padding-top: .05em;",
                      tabularray_inner = "rowsep={.05em}, colsep = {.1em}")
```

The final column of @tbl-post-exclusion-balance shows the same analysis as the first column but including the previously excluded respondents (i.e. those who failed the attention check or went too fast/slow). Balance does not improve, suggesting that to the extent that there is imbalance it is not due to the exclusions.  


\clearpage

```{r main-no-exclusions}
#| echo: false
#| label: tbl-main-results-w-exclusions
#| tbl-cap: "Effect of information treatments on support for emission reduction policies (with sample exclusions)"
spi_reg_list_w_exclusions <- spi_reg_list_given_data(D_post_exclusions)
main_result_table_from_reg_list(spi_reg_list_w_exclusions) |> 
  tinytable::style_tt(fontsize = .75) 
 #                     bootstrap_css = "padding-right: .05em; padding-top: .05em;",
#                      tabularray_inner = "rowsep={.05em}, colsep = {.1em}")
```


```{r}
#| label: tbl-het-main-w-exclusions
#| tbl-cap: "Heterogeneity in effect of SAI information on support for emission reduction policies (with sample exclusions)"
spi_het_reg_list_w_exclusions <- spi_het_reg_list_given_data(D_post_exclusions)
het_reg_table_given_reg_list(spi_het_reg_list_w_exclusions)
```


```{r main-mechanisms-w-exclusions}
#| echo: false
#| label: tbl-sai-on-other-outcomes-w-exclusions
#| tbl-cap: "Effect of SAI information on other attitudes (with sample exclusions)"

other_attitudes_rl <- other_attitudes_reg_list(D_post_exclusions)
other_attitudes_table_from_reg_list(other_attitudes_rl)
```

```{r}
#| echo: false
#| label: tbl-other-outcomes-by-treatment-arm-w-exclusions
#| tbl-cap: "Effect of each information treatment on other attitudes (with sample exclusions)"

reg_list_other4 <- reg_list_other_outcomes4(D_post_exclusions)
table_for_other_outcomes4(reg_list_other4)
```


```{r}
#| echo: false
#| label: tbl-pre-reg-report
#| tbl-cap: "Pre-registration reporting table"

tribble(~` `, ~`Specified in pre-registration?`, ~`Reported in manuscript?`, ~`Deviations`,
        "Sampling", "Yes", "Yes (pp 9-10)", "",
        "Sample exclusions", "Yes", "Yes (pp 9-10)", "Results using original exclusion plan appear in Appendix; manuscript explains that we analyze the full sample to satisfy journal requirements (explained pp 9-10 in manuscript)",
        "Experimental conditions", "Yes", "Yes (pp 7-8)","",
        "Observed measure 1: Normalized index of support for emissions reduction policy", "Yes", "Yes (pg 7)", "",
        "Observed measures 2-5: Worry about climate change, perceived threat from tech responses to CC, belief in efficacy of emissions reductions, belief in 'techno fix'", "Yes", "Yes (pp 8-9)", "",
        "Hypotheses", "NA", "NA", "Our PAP contains no hypotheses. We do specify a measurement strategy for several causal estimands. See next several rows.",
        "Measuring effect of SAI information on index of policy preferences", "Yes", "Yes (section 5.2)", "None",
        "Measuring heterogeneous effects of SAI information on index of policy preferences", "Yes", "Yes (section 5.3)", "None",
        "Measuring effect of SAI information on other attitudes", "Yes", "Yes (section 5.4)", "None",
        "Were there studies included in pre-registration that are not reported in the manuscript? (If yes, explain)", "No", "NA", "") |> 
  tinytable::tt(width = c(2,1,1,2))


```


